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LONG-TERM  GOAL 

The  long  term  goal  is  development  and  utilization  of  a  suite  of  high-resolution,  linescan-camera  based 
imaging  systems  to  investigate  the  distribution  and  composition  of  plankton  and  suspended  particles  in 
the  marine  environment.  The  system  is  designed  with  wide  dynamic  range  and  will  ultimately  be  used 
for  high  speed,  high  resolution  characterization  of  water  column  particle  fields  in  high,  medium  and 
low  latitudes.  Such  a  system  will  have  broad  application  to  areas  of  defense  and  environmental 
interests  such  as  ecological  modeling  efforts,  Autonomous  Ocean  Sampling  Networks  (AOSN),  and 
ground-truthing  of  satellite  data. 

OBJECTIVES 

The  project  has  two  major  objectives.  The  first  is  the  development  of  high  resolution  imaging 
instruments  for  use  in  in-situ  sampling  of  plankton  and  particle  distribution  in  the  upper  200  meters  of 
the  ocean.  The  second  major  objective  is  to  test  and  apply  the  developed  instrumentation  to  support 
scientific,  defense,  and  environmental  applications  toward  the  development  of  biophysical  models  for 
the  spatial  distribution  of  particles  and  zooplankton. 

APPROACH 

The  scientific  approach  for  this  project  is  to  sequentially  continue  development  and  testing  of  the 
Shadowed  Image  Particle  Profiling  and  Evaluation  Recorder  (SIPPER)  in-situ  particle  imaging  system, 
and  to  analyze  technical  and  scientific  results  arising  from  its  deployment.  The  system  is  designed  to 
image  zooplankton  in  the  size  range  of  a  few  tens  of  microns  to  over  1  centimeter  in  size.  It  was 
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designed  for  mounting  on  the  previously  ONR  supported  High  Resolution  Sampler  (HRS)  [Sutton  et 
ah,  2001],  SIPPER  currently  consists  of  a  pair  of  high-speed  digital  linescan  cameras,  and  light  source 
and  optics  for  back  illuminating  and  imaging  the  particles,  and  a  custom  high-speed  data  compression 
and  storage  subsystem  for  shipboard  control,  feedback,  and  storage.  SIPPER  has  been  continually 
improved  incrementally  as  technology  improvements  became  available  (for  instance  the  current 
cameras  have  3  times  the  imaging  resolution  as  the  original  ones  used).  Custom  underwater  high-speed 
data  compression  and  storage  electronics  were  developed  to  handle  the  data  rates.  Continuing  this 
approach,  this  year  we  have  initiated  a  redesign  and  combining  of  the  SIPPER  and  MIPPER  (miniature 
imaging  volume  version  of  SIPPER),  incorporating  the  key  components  of  the  previous  system,  while 
refreshing  and  improving  on  its  limitations.  These  improvements  will  be  discussed  in  detail,  below. 

WORK  COMPLETED 

Sensor  Development 

Although  we  originally  proposed  primarily  field-testing  of  the  instrument  in  FY03,  the  realization  was 
made  that  new  off-the-shelf  technologies  could  enable  several  significant  improvements  to  the 
instrument.  This  would  serve  the  ultimate  engineering  goal  of  the  project  to  create  a  particle  imaging 
device  which  has  a  wide  range  of  resolutions,  is  robust,  has  low  power  requirements,  is  simple  to  use 
and  repair  and  is  small  enough  to  fit  on  most  AUVs  and  towed  platforms.  A  reallocation  of  funds 
request  was  granted  to  allow  these  developments  to  proceed. 

The  new  design  is  called  SIPPER3,  as  it  is  the  third  iteration  of  the  original  SIPPER.  SIPPER3 
incorporates  a  flexible  optical  configuration  that  can  be  optimized  either  for  imaging 
mesozooplankton-sized  particles  sampled  in  the  original  100  mm  square  sampling  tube  (SIPPER 
configuration)  or  for  imaging  the  smaller,  more  abundant  plankton  groups  using  a  much  reduced 
imaging  area  (MIPPER  configuration).  Additionally,  the  SIPPER3  electronics  have  been  simplified, 
and  overall  size  reduced  by  90%  compared  to  SIPPER2  (Figure  1).  These  improvements,  and  its 
simplification  to  a  single  camera  system  should  significantly  lower  its  final  cost,  promoting  widespread 
use.  Major  changes  being  incorporated  into  SIPPER  3  are  summarized  below. 

Single-Camera  System 

In  order  to  reduce  system  cost  and  complexity,  it  was  decided  to  make  SIPPER3  a  single-camera 
system.  Most  researchers  have  not  indicated  a  significant  interest  in  SIPPER2’s  ability  to 
simultaneously  image  particles  from  two  orthogonal  directions.  To  accommodate  those  who  would 
want  a  two-camera  system,  SIPPER3  is  designed  so  that  two  separate  SIPPERs  can  be  mounted 
together  and  a  synchronization  signal  can  be  sent  from  one  SIPPER  to  the  other  to  allow  data 
alignment  during  post-processing. 

New  Image  Compression  Board 

By  using  a  new  state-of-the-art  field  programmable  gate  array  (FPGA),  most  of  the  41  integrated 
circuits  on  SIPPER2’s  FPGA  circuit  board  have  been  incorporated  into  a  single,  smaller  chip.  The  flat- 
field  correction  and  image  compression  algorithms  are  the  same  as  those  used  successfully  in 
SIPPER2. 

Migration  to  a  FireWire  data  storage  system 

FireWire®,  also  known  as  IEEE  1394,  is  a  high-speed  serial  bus  that  is  commonly  used  for  digital  video 
cameras  and  external  hard  drives.  This  will  allow  us  to  convert  the  data  storage  system  of  SIPPER3 
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from  3  ultra-ATA  hard  drives  and  a  costly  controller,  to  a  single  120  GB  FireWire®  hard  drive.  This 
drastically  improves  power  consumption,  the  number  of  connectors,  size,  and  speed.  The  data  storage 
system  only  needs  a  6-conductor  cable  to  connect  to  a  swappable  hard  drive. 

Environmental  Sensor  Data  Storage 

Previously,  environmental  data  was  collected  separately  from  SIPPPER  image  data  and  therefore  had 
to  be  time-synchronized  for  analysis.  SIPPER3  allows  the  data  from  other  sensors  to  be  embedded  in 
the  image  data  stream  and  stored  together.  The  method  of  combining  the  data  from  various  sensors  and 
transferring  it  to  SIPPER  is  the  subject  of  a  patent  application  and  cannot  be  disclosed  at  this  time. 

SIPPER2  and  SIPPER3  Size  Comparison 


0.5  m 


Figure  1.  Size  comparison  of  SIPPER  components  between  SIPPER2  and  SIPPER3.  Components 
are  A.  Data  storage,  B.  Sampling  tube,  C.  Camera  and  optics,  and  D.  Illumination  source. 
[Improvements  made  SIPPER3  significantly  smaller  in  size  and  weight  than  SIPPER2] 

New  User  Interface 

SIPPER3  has  a  flexible  user  interface  dependent  on  the  platform  SIPPER  is  deployed  on.  The  most 
commonly  used  interface  is  the  same  as  SIPPER2,  using  an  embedded  web  server  (Lantronix  X-Port) 
as  a  graphical  interface  that  allows  basic  on/off  control  of  the  cameras,  illumination  system,  and 
recording,  as  well  as  display  text  readout  of  some  other  sensors  (temperature,  depth,  flow-rate,  etc.). 
The  second  interface  is  a  serial,  text-based,  menu  system  that  simplifies  control  interfacing  with 
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autonomous  vehicles  or  moorings.  It  also  serves  as  a  backup  interface  in  case  of  Ethernet  network 
failure.  The  preferred  interface  that  is  capable  of  displaying  real-time  SIPPER  imagery  and  sensor  data 
in  a  graphical  format  is  under  development.  For  this,  the  FPGA  on  the  Image  Compression  Board  will 
collect  “snapshot”  sample  images  and  send  them  to  the  GUI  program  on  the  shipboard  computer  to 
display  to  the  operator  once  per  second. 

New  Optical  Design 

The  SIPPER3  design  uses  a  multi-lens  system  that  has  shown  less  distortion  in  computer  modeling, 
while  allowing  a  shorter  path  length.  The  laws  of  physics  limit  the  ratio  of  depth-of-field  vs.  resolution 
that  is  also  dependent  on  the  wavelength  of  light.  An  adjustable  iris  will  be  positioned  to  allow  this 
tradeoff  to  be  easily  adjusted,  although  different  sampling  tubes  will  be  needed  to  keep  the  center  of 
focus  in  the  middle  of  the  tube.  This  feature  allows  SIPPER3  to  fit  the  requirements  of  SIPPER  and 
MIPPER,  interchangeably. 

A  new  illumination  system  uses  high-intensity  LEDs  to  reduce  size,  complexity,  and  power 
consumption,  yet  increase  reliability.  LEDs  are  available  in  a  range  of  colors,  allowing  us  to  select  a 
short  wavelength  (504nm  cyan)  to  achieve  better  resolution  than  with  the  635nm  laser  illumination 
used  in  prior  SIPPER  versions. 

New  Instrument  Layout  and  Assembly 

Enabled  by  the  reduction  in  the  size  of  the  SIPPER3  image  compression  and  data  storage  electronics, 
we  chose  to  house  both  boards  in  the  same  pressure  vessel  as  the  camera  and  imaging  optics,  to  reduce 
the  complexity  (and  failure  potential)  of  the  external  electrical  connectors.  The  reduced  optical  path 
length  eliminates  the  need  for  the  folded  light  path  used  in  SIPPER  1  and  SIPPER2.  The  folded  light 
path  required  a  window  and  mirror-mounting  block  to  be  welded  into  the  tube.  This  made  it  difficult 
to  install  and  adjust  the  optics.  These  improvements  will  save  weeks  of  assembly,  adjustment  and 
calibration  time. 

Field  Application 

SIPPER2  was  deployed  on  the  High  Resolution  Sampler  [Sutton  et  al.,  2001]  for  a  planned  ten-day 
cruise  in  the  northeast  Gulf  of  Mexico  on  July  10,  2003.  Our  goal  was  to  use  SIPPER  to  investigate  the 
horizontal  and  vertical  distribution  patterns  of  oceanic  mesozooplankton  and  other  particles  in  the 
presence  of  a  narrow  band  of  Mississippi  River  plume  water  that  had  been  advected  offshore.  This 
cruise  ended  prematurely  when  the  custom  made  block  used  to  deploy  the  HRS  off  the  ship  U-frame 
catastrophically  failed  during  the  first  day  of  deployments.  Approximately  100  minutes  of  new 
SIPPER  grayscale  data  was  collected  before  the  mechanical  failure.  No  instruments  were  damaged.  A 
new  block  has  been  ordered  and  shiptime  has  been  rescheduled. 

RESULTS 

Analysis  of  SIPPER  data  collected  in  the  northeast  Gulf  of  Mexico  during  the  summer  of  2000  indicate 
that  SIPPER  can  provide  significantly  more  accurate  estimates  of  mesozooplankton  abundance, 
biomass  and  composition  than  either  an  Optical  Plankton  Counter  (OPC)  or  plankton  nets  [Remsen  et 
ah,  in  press].  This  study  required  manual  classification  of  over  175,000  SIPPER  images  into  14 
plankton  and  particle  classes. 
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As  this  is  both  a  labor  intensive  and  expensive  effort,  much  of  this  years  work  has  been  spent  on  a 
parallel  effort  with  investigators  from  another  ONR  funded  project  (Award  #  N000 14-02-0266)  to 
develop  an  automated  image  classification  software  package  to  use  with  SIPPER.  Our  contribution  has 
entailed  extraction  of  binary  and  grayscale  images  from  multiple  collected  SIPPER  deployments,  and 
classification  of  the  images  into  identifiable  groups.  Over  100,000  images  have  been  manually  sorted 
into  over  50  plankton  and  particle  classes  and  grouped  into  experimental  training  sets.  The  sorted 
images  have  been  used  as  initial  validation  data  for  classification  models  that  are  being  tested  in  lab 
and  field  use. 

The  new  extraction  and  classification  software  package  was  tested  at  sea  and  perfonned  well.  Images 
could  be  extracted,  classified  and  written  to  disk  almost  as  fast  as  the  data  was  generated.  For  example, 
an  8-minute  SIPPER  run  at  20  meters  had  12,000  particle  images  extracted  and  classified  in  14 
minutes. 

The  transition  to  grayscale  imaging  has  allowed  for  significant  improvement  to  our  automated 
classification  effort.  Preliminary  experiments  indicate  that  addition  of  grayscale  features  has  improved 
the  overall  classification  model  accuracy  by  over  10%. 

IMPACT/APPLICATIONS 

This  project  represents  a  directed  effort  to  develop,  build  and  deploy  imaging  systems  that  can 
characterize  the  oceanic  particle  spectrum  in  sizes  from  centimeters  to  micrometers  and  be  deployed  on 
AUVs,  ROVs  and  towed  or  moored  sampling  systems.  As  such,  it  is  a  part  of  the  larger  goals  of  much 
ONR-directed  research  to  create  systems  that  can  efficiently  monitor  oceanic  conditions  with  a 
minimum  of  temporal  and  spatial  degradation  and  without  the  use  of  expensive  and  time-consuming 
research  vehicles.  Such  information  would  then  become  a  major  input  to  forecasting  (or  ‘nowcasting’) 
significant  conditions  in  the  coastal  ocean. 

RELATED  PROJECTS 

Development  of  software  enabling  automated  classification  of  SIPPER  images  is  being  funded  through 
ONR  award  #  N00014-02-1-0266.  The  SIPPER  web  site  is  at  http://marine.usf.edu/sipper. 
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